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Abstract 

This study examines reported best and worst supervisee experiences of participants. Most supervision re- 
search emphasizes productive and effective aspects of supervision. Some efforts, however, have empha- 
sized nonproductive or lousy aspects of supervision (Worthington, 1987). A factor analysis of participants’ 
reported best supervisee experience revealed a one-factor solution, a finding consistent with qualitative re- 
search efforts suggesting integration of roles and expectations is key for effective supervision. By contrast, 
a factor analysis for participants’ reported worst experiences as a supervisee revealed a two -factor solution, 
suggesting a lack of integration as well as problems in two distinct areas of supervision practice. Conclu- 
sions and suggestions for further research are offered. 


Supervision in the helping disciplines is a critical component of professional development (Ronnestad & 
Skovholt, 1993). Supervision provides the experiential foundation for integrating, with increased compe- 
tence, theoretical principles into practice (Bernard & Goodyear, 1998; Falender & Shafranske, 2005; Ron- 
nestad & Skovholt). Some authors have emphasized aspects of the supervisory process associated with 
exemplary supervisory behaviors as well as strategies characteristic of effective supervision (Borders, Cash- 
well, & Rotter, 1995; Ronnestad & Skovholt). The most prominent elements of supervision emphasize: (a) 
structure (e.g., clarifying role expectations, intentionality, and deliberate planning) (Bradley & Ladany, 
2001; Ladany, Friedlander, & Nelson, 2005; Nelson & Friedlander, 2001; Ronnestad & Skovholt; Vespia, 
Heckman- Stone, & Delworth, 2002), (b) strategy (e.g., compromise, tolerance, and scrutiny emphasizing 
both support and challenge) (Falender & Shafranske), or (c) relationships (e.g., empathic understanding, 
genuineness, respect, and concreteness) (Blocher, 1983; Carifio & Hess, 1987; Veach, 2001; Worthen & 
McNeill, 1996). 

Nonproductive aspects of supervision have received far less attention in the literature. As Worthington 
(1987) observed, “a good theory of lousy supervisor behaviors is missing” (p. 203). Worthington alluded to 
the potential value of examining supervisor behaviors indicative of nonproductive or even detrimental su- 
pervision. Similarly, Ladany, Friedlander, and Nelson (2005) observed that problematic behavior of super- 
visees has elicited intense scrutiny, yet supervisor incompetence has been virtually ignored in the literature. 



An initial theoretical framework to conceptualize lousy supervision emerged out of a qualitative study con- 
ducted by Magnuson, Wilcoxon, and Norem (2000). These investigators sought to identify nonproductive 
supervisory behaviors and to develop a schema for categorizing such behaviors. Magnuson et al. identified 
three primary domains of ineffective supervisory behaviors: (a) administrative/organizational, (b) cognitive/ 
technical, and (c) relational/affective. Indicators of inadequate supervision in the administrative/ 
organizational domain primarily reflect the supervisor’s level of adeptness in structuring the supervisory 
process, clarifying expectations and objectives, and managing the ongoing procedures associated with su- 
pervision. In the cognitive/technical domain, professional competence as a practitioner and as a supervisor 
is the focal point for perceived supervisor inadequacy. Further, deficits in professionalism accompanied by 
vague and abstract feedback from supervisors, supervisor reliance on a single theoretical orientation, or un- 

The Alabama Counseling Association Journal, Volume 35, Number 2 




willingness to incorporate supervisee theoretical ideas tend to amplify perceptions of supervisor inadequacy. 
Examples of inadequate supervision in the affective/relational domain include limited interpersonal sensitivity, 
compromised confidentiality, and failure to create a safe environment. 


Magnuson et al. (2000) cautioned that lousy supervision may not be easily defined. They concluded that 
“lousy supervision is more complex than either the presence of ineffective practices or the absence of effective 
practices” (p. 200). Instead, they suggested that “lousy supervision seems to result from a combination of both 
factors, with lousy supervision anchoring one extreme of a continuum that ranges to an opposite and equally 
complex construct of excellent supervision” (p. 200). Such an observation is consistent with the efforts of 
Allen, Szollos, and Williams (1986), who concluded that nonproductive supervision is difficult to clarify 
based on the presence of specific characteristics because no single characteristic is a critical determinant of 
poor supervision. Essentially, poor supervision appears to be described best by what supervisors fail to pro- 
vide, rather than what they provide. Watkins (1997) expressed a similar belief that ineffective supervision 
may not be the exact opposite of effective supervision. 

Various demographic variables have received attention concerning their influence on supervisee -supervisor 
interactions and their contributions to the formation of supervisory relationships. These variables include gen- 
der, race/ethnicity, supervisee-only status versus supervisor-supervisee status, and academic affiliation 
(Borders, 2001; Cook, 1994; Helms, 1990). The impact of these and related variables on the supervisory rela- 
tionship is likely to be “subtle and highly complex” (Nelson & Holloway, 1990, p. 478). Some researchers 
have argued that designations of gender and racial/ethnic identity are overly simplistic and insufficient in culti- 
vating a deep understanding and appreciation of an individual’s cultural formation and its impact upon the 
supervisory process (Ellis & Ladany, 1997; Fong & Lease, 1997; Nelson, Gray, Friedlander, Ladany, and 
Walker, 2001). However, some categorical variables appear to be relevant in relation to lousy or nonproduc- 
tive supervision. Specifically, two such variables are (a) supervisor experience (i.e., supervisee -only or super- 
visee and supervisor status), and (b) supervision context (i.e., an academic experience as a component of 
graduate study or post-graduate supervision in an applied setting). 

Skovholt and Jennings (2004) observed “To be an expert, one must accumulate experiences that deepen, im- 
prove, and extend one’s vision of a given field or discipline” (p. 4). In terms of supervisory experience, Fong, 
Borders, Ethington, and Pitts (1997) and Granello (2002) suggested that the most compelling gains in higher- 
order counseling expertise emerge when supervisees experience the role of supervisor. Peace and Sprinthall 
(1998) asserted that understanding the complexity of both a counselor and a supervisor role promotes maturity 
and greater appreciation for the tasks and demands of supervision. Thus, familiarity with the roles of supervi- 
see and supervisor could inform one’s appraisals of their supervised experiences as being beneficial or nonpro- 
ductive. 

Investigating problematic supervision would appear to be of value insofar as such findings might increase un- 
derstanding about what not to do in clinical supervision. Given the inherent complexity involving the con- 
structs of good supervision and lousy supervision, attempts to examine commonalities as well as differences at 
both ends of the spectrum of clinical supervision could prove beneficial in understanding more fully the super- 
vision process. Moreover, given the increasing emphasis placed on the quality control aspect of clinical super- 
vision by licensure and credentialing agencies along with training program accrediting bodies (Bernard & 
Goodyear, 1998), that only one study could be found in the professional literature investigating best and worst 
supervision experiences suggests the timeliness of such an effort. Additional study juxtaposing best and worst 
supervision experiences may contribute to greater clarity and understanding of factors that distinguish effec- 
tive from ineffective supervision. 

Magnuson and Wilcoxon (1998) observed that supervision in graduate school differs from supervision in post- 
graduate practice. For example, counselors -in-training are enrolled in an academic program with built-in 
structure, well-defined curricula, and typically have multiple supervisors at any point in time. Supervisors of 
prelicensed counselors, however, assume sole responsibility for the supervisee without the benefit of academi- 
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cally defined evaluation criteria or any prior unbiased assessments of the supervisee’s strengths and weak- 
nesses. Furthermore, academic supervisors are able to closely monitor the work of the student supervisee, 
whereas the supervisor of a prelicensed counselor may have limited contact and limited opportunities to ob- 
serve a supervisee’s work. Additionally, while student supervisees pay tuition, in post-academic supervision 
there is a direct fee-for-service transaction that takes place between supervisors and supervisees. Finally, 
prelicensed supervisees are free to choose their supervisor. Each of these aspects of the supervisory relation- 
ship has implications for the process and outcome of the supervisory experience and the nature of the supervi- 
sory relationship, both in terms of exemplary and nonproductive supervisory practices. 

Results and discussion will be offered concerning distinctions in factor patterns distinguishing between pro- 
ductive and nonproductive supervisory practices with respondents. The initial purpose of this study was to 
develop and validate an instrument to determine whether the domains of lousy supervision identified by 
Magnuson et al. (2000) could be validated quantitatively. A second purpose was to determine if the three do- 
mains of supervision reflect similar factor patterns for participants rating their best supervision experience and 
their worst supervision experience. A third purpose was to examine the influence of selected demographic 
variables on the participants’ ratings. Specifically, the variables examined in this element of the study con- 
cerned supervisor experience and the contextual setting of supervision (i.e., graduate/academic or post- 
graduate). 

Method 


Participants 

The population for this study was professional members of the American Counseling Association (ACA) who 
had completed a minimum of 5 years of postgraduate practice as a counselor and who had participated in face- 
to-face supervision at some point in their graduate or postgraduate careers. The professional membership level 
represents the highest level of membership, distinguished from associate or student status in the ACA. A na- 
tional random sample of 1,000 participants was selected. No other screening criteria were used, although the 
hope was that random selection would yield a sample reflecting an array of practice settings, professional ex- 
perience as a counselor, and supervisor experience. 

Of the 1,000 questionnaire packets distributed, 290 questionnaires were returned, for a response rate of 29%. 
However, 12 of the questionnaires could not be used because they were either incomplete or were completed 
by a participant who did not meet the eligibility criteria for inclusion. A total of 278 usable questionnaires 
yielded a 27.8% response rate. 

Instrumentation 

A survey instrument was designed to elicit information regarding various aspects of supervisor behaviors. 
Items were designed to reflect supervisory behaviors that typically occur in one of the three domains of lousy 
supervision (administrative/organizational, cognitive/technical, and relational/affective) identified by 
Magnuson et al. (2000). The survey was a Likert -type scale consisting of 19 items. 

To establish construct validity prior to data collection for the study, the instrument was distributed to five 
counseling professionals who agreed to serve as expert raters (four counselor educators and one student affairs 
administrator). All raters possessed a doctorate in counselor education, and each held a credential for clinical 
supervision in counseling. As a group, the number of years involved in the practice, teaching, or research of 
counselor supervision ranged from 5 to 30. The raters reported that during their careers the number of practi- 
tioners/students they supervised ranged from 15 to over 300. The raters also reported having had between two 
and six clinical supervisors each during their professional lives. 

The expert raters were asked to consider each item of supervisory behavior in the instrument and to determine 
which of the three domains of supervision (i.e., administrative/organizational, technical/cognitive, and rela- 
tional/affective) each item best fit. A tally of the rater responses revealed that on 1 1 of the 19 items of super- 
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visory behaviors, rater agreement was 100%. Rater response on two items revealed little agreement among the 
raters, thus these two items were discarded. Based on expert feedback, two items (“Gave too much or too lit- 
tle corrective feedback”) and (“Gave too much or too little affirming feedback”) were amended so that each 
item reflected a single inquiry (e.g.. Gave too much corrective feedback,” “Gave too little corrective feed- 
back,” etc.). This final revision, designated as the Supervisory Behavioral Profile (SBP), yielded an instru- 
ment with a total of 19 items. 

Procedure 

Survey packets were distributed by postal mail to the preferred mailing addresses indicated on participants’ 
ACA membership profile. No distinction was made concerning home or work designations for survey pack- 
ets. Each packet contained an introductory letter explaining the purpose of the research along with a statement 
of confidentiality and informed consent, a form for reporting demographic data, two versions of the SBP, and 
a self-addressed stamped envelope for return. 

Participants were asked to complete two versions of the SBP. The first version (SBP-Best) inquires about “my 
best supervisory experience,” and the second version (SBP-Worst) inquires about “my worst supervisory ex- 
perience.” Responses to items on both versions range from 1 {Strongly Disagree) to 4 {Strongly Agree). Spe- 
cifically for the SBP-Best, each item represents a response to the sentence stem, “As a description of my best 
supervisor. ...” Lower scores on the SBP-Best represent supervisee perceptions of productive supervisor 
behavior. For the SBP-Worst, each item represents a response to the sentence stem, “As a description of my 
worst supervisor ...” Higher scores represent supervisee perceptions of nonproductive supervisor behavior. 
Factor analyses were conducted to compare group data for participants rating their best and worst supervisory 
experiences. 

A short demographic data form was included with the survey questionnaires to gather descriptive data about 
the participants and about their supervisors. These data included age, gender of participant, race/ethnicity of 
participant, gender of best and worst supervisor, and race/ethnicity of best and worst supervisor. Two addi- 
tional variables of interest were also included on the demographic form. First, participants were asked to in- 
dicate their experience in a supervisory role. Specifically, participants were queried as to whether they had 
served in the role of supervisee only or in the role of both supervisee and supervisor. Secondly, participants 
were asked whether their best and worst experiences as a supervisee occurred as a graduate student or in their 
post-graduate status. A series of ANOVA’s were conducted to determine the impact of selected demographic 
variables on nonproductive supervision. 

All procedures were reviewed and approved by the Institutional Research Board (IRB) of The University of 
Alabama. No follow-up reminders or subsequent contact was made with participants to encourage higher rates 
of participation. 


Results 


Participants 


Participant ages ranged from 30 to 79 years with a mean age of 52 and a median age of 53. Of the 267 partici- 
pants who provided information regarding gender, 57 were male and 213 were female. Two hundred forty- 
eight were Caucasian, 10 were African American, 7 were Hispanic American, 5 selected “other,” and 5 did not 
respond to this inquiry. Seventy-one participants reported having a doctoral degree in counseling and 193 
reported having a master’s degree in counseling. 
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The mean number of years of counseling experience among the participants was 17.1. When asked about cer- 
tification status, 256 participants indicated they were certified, licensed, or both, while 18 reported no such 
credential. 

The participants were asked to indicate their experience in a supervisory role. Specifically, participants were 
queried as to whether they had served in the role of supervisee only or in the role of both supervisee and su- 
pervisor. Seventy-seven participants indicated they had only served in the role of a supervisee, while 194 re- 
ported they had served as both supervisee and supervisor. Of the participants, 128 stated they had not taken 
had taken a graduate supervision course, while 142 indicated that they had taken such a course. 

Another variable of inquiry was whether the participants’ best and worst experiences as a supervisee occurred 
as a graduate student or in a post-graduate status. One hundred twenty-seven reported their best supervisory 
experience occurred during graduate study, while 144 reported their best experience occurred in a post- 
graduate context. By contrast, 125 indicated their worst supervisory experience occurred during graduate 
study, and 136 indicated their worst supervisory experience occurred in a post-graduate context. 

One hundred twenty-seven reported their best supervisory experience occurred during graduate study, while 
144 reported their best experience occurred in a post-graduate context. By contrast, 125 indicated their worst 
supervisory experience occurred during graduate study, and 136 indicated their worst supervisory experience 
occurred in a post-graduate context. 

Data Analysis 

Factor analyses were conducted to identify factor patterns related to participants’ ratings of their best supervi- 
sion experience and their worst supervision experience. Data were derived from participant responses on the 
SBP-Best and SBP-Worst. The results of these factor analyses are displayed in Tables 1 and 2. 

For the SBP-Best, principal components factor analysis yielded three factors with a Kaiser-Guttmen criterion 
of eigenvalue greater than 1.00. Because the criterion of eigenvalue greater than 1.00 may misjudge the most 
appropriate number of factors (Gorsuch, 1983), the scree plot was carefully examined. The scree plot indi- 
cated the total variance was best explained by a single-factor solution for the SBP-Best factor analysis. 

The principal component factor analysis conducted on the SBP-Worst yielded a five-factor solution based on 
the Kaiser-Buttmen criterion of eigenvalue greater than 1.00. Examination of the scree plot indicated a two- 
factor solution was the most appropriate solution. Therefore, a varimax rotation was conducted using a two- 
and three-factor model. The two-factor solution was adopted because it provided the most distinct structure 
for ease of interpretability. Factor 1 was designated as “Relational” because the majority of the items and fac- 
tor loadings reflected some aspect of the interpersonal relationship between supervisee and supervisor. Factor 
2 was designated as “Technical/Cognitive” because the majority of the items reflected aspects of the supervi- 
sory process that were cognitively based or technical in nature. Table 2 presents the two factors with their 
respective items and factor loadings. Factor 1 consists of 1 1 items, accounting for 26.6% of the total variance. 
Factor 2 consists of 8 items, accounting for 19.43% of the total variance. Data generated from participant re- 
sponses to the SBP-Best and SBP-Worst analyses reflected distinctly different factor patterns between partici- 
pants rating their best supervisory experience when compared to their worst supervisory experience. The 
reliability of both instruments was also assessed. The alpha coefficient for the SBP-Best was .92, and the al- 
pha coefficient for the SPB -Worst was .87. 

Because the emphasis of this study was on lousy supervision, ANOVAs were conducted to determine the in- 
fluence of supervision context on SBP-Worst scores. The first ANOVA compared total scores, which could 
range from 19 to 76. Respondents who had been both a supervisor and supervisee had more negative percep- 
tions of their worst supervision experience {M = 49.82, SD = 9.75) than those who had only been a supervisee 
46.53, SD = 1 1.36), 7^(1,243) = 4.92, < .05. Comparisons were also made by Relational and TechnicaF 
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Cognitive subscores. The possible range of Relational scores was from 10 to 44. A significant difference 
was found on the Relational subscale when comparing counselors whose worst supervision experience was 
as after completing graduate school {M= 29.76, SD = 6.94) with those whose worst supervision experience 
occurred while in graduate school (M= 27.53, SD = 7.07), 7^(1,246) = 5.69, p < .05. The counselors whose 
worst supervision experience occurred after completing graduate school had more negative perceptions of 
their supervisors. 



The possible range of Cognitive/Technical scores was from 9 to 36. A significant difference was found 
when comparing the Cognitive/Technical scores of participants who had both a supervisee and a supervisor 
17.26, SD = 3.95) with those who had been only a supervisee 18.54, SD = 4.11), T’(l,249) = 4.98, 
p < .05. The counselors who had been only a supervisee had more negative perceptions of their worst super- 
vision experience than those who had been both a supervisee and a supervisor. 


Table 1 

Factor Analysis Results for Supervisory Behavioral Profile - Best 


Item 


Factor 1 


Showed insensitivity to my developmental needs .787 

Showed insensitivity to my professional development needs (e.g., .745 

lack of professional role modeling, lack of commitment to 
my professional growth) 

Failed to appreciate my theoretical model or orientation .718 

Gave too little affirming feedback .715 

Provided vague feedback .715 

Was an unskilled supervisor .712 

Avoided issues that arose between the two of us .710 

Was an unreliable professional resource (e.g., used supervision time .691 

to provide therapy, confidential information disclosed) 

Gave too little corrective feedback .690 

Imposed his or her personal agenda .687 

Gave too much corrective feedback .668 

Failed to provide standards for accountability (e.g., expectations not .628 

clarified, evaluative criteria not articulated) 

Was an unskilled practitioner .591 

Gave too much affirming feedback .587 

Focused primarily on microskills and techniques .580 

Was intrusive (e.g., micromanaging, controlling) .570 

Failed to clarify expectations for supervision .566 

Relied on a single primary theoretical model of counseling .537 

Failed to provide a safe environment during supervision .503 
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Table 2 

Factor Analysis Results for Supervisory Behavioral Profile - 

Worst 


Item 

Factor 1 

Factor 2 

Gave too much corrective feedback 

.790 

-.073 

Was intrusive (e.g., micromanaging, controlling) 

.764 

.010 

Failed to appreciate my theoretical model or orientation 

.724 

.134 

Imposed his or her personal agenda 

.695 

.303 

Showed insensitivity to my developmental needs 

.681 

.411 

Gave too little affirming feedback 

.678 

-.034 

Focused primarily on microskills and techniques 

.644 

.027 

Relied on a single primary theoretical model of counseling 

.568 

.074 

Showed insensitivity to my professional development needs 

.543 

.481 

(e.g., lack of professional role modeling, 
lack of commitment to my professional growth) 

Failed to provide a safe environment during supervision 

.492 

.284 

Avoided issues that arose between the two of us 

.399 

.391 

Failed to provide standards for accountability (e.g.. 

.075 

.743 

Expectations not clarified, evaluative 
criteria not articulated) 

Gave too little corrective feedback 

-.280 

.688 

Failed to clarify expectations for supervision 

.174 

.638 

Was an unskilled supervisor 

.312 

.644 

Provided vague feedback 

.075 

.635 

Was an unskilled practitioner 

.291 

.509 

Was an unreliable professional resource (e.g., used super- 

.364 

.503 

vision time to provide therapy, confidential 
information disclosed) 

Gave too much affirming feedback 

-.219 

.415 



Discussion 

The initial purpose of the study was to develop and validate an instrument to determine whether the domains 
of lousy supervision identified by Magnuson, Wilcoxon, and Norem (2000) could be validated quantitatively. 
The findings did not provide validation of the three domains of lousy supervision identified by Magnuson et 
al. (2000). One possible explanation could be that the instrument used in the present study lacked a sufficient 
number of items to represent each domain of lousy supervision. Additionally, Magnuson et al.’s findings may 
not have been confirmed because the respondents participating in their study were exclusively classified as 
experienced supervisors, while a portion of the participants in the present study (27.8%) had never served as 
supervisors. The data did, however, identify two domains of nonproductive supervision that appear to corre- 
spond to the relational/affective domain and the technical/cognitive domain as described by Magnuson et al. 
Further, the data were consistent with previous findings reported by Gray, Ladany, Walker, and Ancis (2001) 
and Ellis (2001), who noted multiple and multifaceted aspects of nonproductive supervision in their outcome 
measures with student supervisees. 


A second purpose of this study was to determine if these three domains of supervision reflect similar factor 
patterns for participants rating their best supervision experience as well as their worst supervision experience. 
The findings suggest that good supervision is different from poor supervision. Magnuson et al.’s (2000) 
speculation that lousy supervision may not be straightforward and easily defined corresponds to the evidence 
found in this study based on different factor patterns for “besf ’ versus “worst” supervisors. In this regard, the 
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complexity involved in defining the constructs of good supervision versus lousy supervision underscores Wat- 
kins’ (1997) observation that ineffective supervision may not be the exact opposite of effective supervision. 


A recurring theme in the supervision literature suggests that productive supervision is associated with effec- 
tively managing multiple tasks that are administrative and relational in nature (Bordin, 1983; Cohen & De- 
Betz, 1977; Holloway, 1995). One context for examining this theme emerges from the discrimination model 
of supervision (Bernard, 1979). This model depicts three possible supervision foci (i.e., intervention skills, 
conceptualization skills, and personalization skills) used in conjunction with three possible supervisory roles 
(i.e., teacher, counselor, and consultant). From this context, a one-factor solution as the preferred representa- 
tion of data from participants describing their best supervisor suggests that effective supervisors can navigate 
smoothly among the differing roles and functions of supervision relative to administrative and relational ele- 
ments of supervision. By contrast, the two-factor solution to represent the profile of participants’ worst super- 
visors could be viewed as a failure of supervisors to integrate and manage the multiple functions and foci of 
supervision to yield productive outcomes. The findings from the current study appear to suggest that failure to 
effectively manage these aspects of supervision could be a critical aspect of nonproductive supervision. 

A third purpose of the study was to examine the influence of selected demographic variables on the factor pat- 
terns for participants’ ratings. Statistically significant differences emerged in the categories of supervision 
setting and supervisor experience. On the Relational subscale, data revealed that participants describing their 
worst supervisory experiences occurred following their graduate study. Although this finding is somewhat 
challenging to interpret due to the scarcity of professional literature addressing the issue of academic versus 
postgraduate supervision, a qualitative study conducted by Magnuson and Wilcoxon (1998) provided a possi- 
ble link. Magnuson and Wilcoxon’s study investigated needs and practices associated with clinical supervi- 
sion of prelicensed (i.e., postgraduate) counselors. Supervisor ambiguity emerged as a prominent theme re- 
flecting uncertainty among participants related to their expectations of supervisor competence. Perhaps the 
findings of this study related to postgraduate supervision may reflect the ambiguity among participants similar 
to that noted by Magnuson and Wilcoxon. 

Two statistically significant findings emerged in the category of supervision status. When compared by over- 
all SBP-Worst scores, the participants who had served in the roles of both supervisee and supervisor held more 
negative views of their worst supervision experience than participants who had only served in the role of su- 
pervisee. This difference was also found when comparing the participants on the Cognitive/Technical sub- 
scale of the SBP-Worst. A possible contributor to this outcome may be that participants with experience as 
both supervisee and supervisor possessed a broader range of experience to draw on when rating their worst 
supervision experience and are therefore able to examine more critically the differences in those supervision 
experiences. Such a conclusion would be consistent with the research conducted by Fong et al. (1997) and 
Granello (2002) that suggests that the most compelling gains in high order counseling skills are brought about 
by the supervisee’s supervised experience, a key component of enhancing cognitive complexity. This view is 
also corroborated by Peace and Sprinthall (1998) in their conclusion that cognitive development should as- 
sume a more critical role in the training of future psychotherapists and should be represented as a specific goal 
of clinical supervision. 

The results of this study should be interpreted with caution and due consideration of its limitations. As an 
initial limitation, the sample was not representative of the population. It was primarily female (77%) and Cau- 
casian (90%). Due to the unequal representation of participants in this study, the results may not be generaliz- 
able to other organizations, ethnicities, or even across genders. As a second limitation, the design of the in- 
strument required participants to respond negatively to negatively phrased items. Such a design may have 
encouraged a test-taking mindset among participants and influenced the nature of their responses, thereby 
compromising the integrity of results. However, despite these limitations, unique elements of the study were 
that participants were recruited via their professional membership status in ACA and that they were profes- 
sionals working in a variety of practice settings from across the US. Additionally, the study represented an 
attempt to examine quantitatively some of the findings previously noted in more qualitative methodologies 
focused on nonproductive supervision. 
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The findings from this study were projected to have implications concerning (a) early identification of supervisors with less 
than adequate performance, (b) early intervention to remediate supervisor, and (c) early intervention to mitigate potential 
negative effects to supervisee (Watkins, 1997). Additionally, the results of this study were projected as a possible contribu- 
tion to greater understanding regarding the distinguishing features of effective and ineffective supervision as a matter of 
supervisor training in counselor education programs. Similarly, quantitative validation of a model of detrimental supervi- 
sory behavior offered potential assistance for (a) counselor educators involved in preparing counselors to supervise, (b) 
practitioners who may be supervising pre-licensed counselors, and (c) practitioners receiving supervision. 
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